Named Entity Recognition Using Conditional Random Fields

Named entity recognition (NER) is an important task in natural language processing, as it widely featured a key information extraction sub-task with numerous application areas. A plethora of attempts was made for NER detection Western and Asian languages. However, little effort has been to develop techniques the Urdu language, which prominent South hundreds millions speakers across globe. considered hard problem owing several reasons, including paucity large, annotated datasets; inaccurate tokenizer; absence capitalization language. To this end, study proposed conditional-random-field-based technique both language-dependent language-independent features, such part-of-speech tags context windows words, respectively. As second contribution, we developed dataset (UNER-I) large number NE types were manually annotated. evaluate effectiveness approach, well usefulness dataset, experiments performed using existing dataset. The results showed that our outperformed baseline datasets by improving F1 scores 1.5% 3%. Furthermore, demonstrated enhanced useful learning prediction supervised approach.